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DETAILED ACTION 

1. This Office action is responsive to the following communication: Amendment filed on 30 
December 2008. 

2. Claims 1-5, 21-24, and 36-39 are pending and present for examination. 

Response to Amendment 

3. Applicant's request for reconsideration of the finality of the rejection of the last Office action is 
persuasive and, therefore, the finality of that action is withdrawn. 

Claim Objections 

4. Claims 1 and 22 are objected to because of the following informalities: the newly added claim 
limitation of "the matching algorithm comprises a plurality factors" should be recited as "the matching 
algorithm comprises a plurality of factors". Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis 
for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use 
or on sale in this country, more than one year prior to the date of application for patent in the United States. 

6. Claims 1-5, 21-24, and 36-39 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Smith et al, "Tools and techniques for color image retrieval" (NPL, hereinafter referred to as SMITH), 
published on March 1996 by Proc. SPIE Vol. 2670, p. 426-437, in view of Barber et al (U.S. Patent No. 
5,579,471, hereinafter referred to as BARBER), filed on 23 March 1994, and issued on 26 November 
1996. 
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7. As per independent claims 1 and 22, SMITH, in combination with BARBER, discloses: 

An image processing system for searching images on a network, the image process 
system comprising: 

(a) a search engine {See SMITH, page 3, subsection D, wherein this reads over "[t]he query is 
communicated to the VisualSEEk server on the World Wide Web through the Common Gateway 
Interface (CGI). The server answers the user's query by accessing the extracted meta-data that 
describes the images and videos in the archive"}; 

(b) an image analyzer coupled to said search engine, said image analyzer for 
comparing first and second images provided thereto from said search engine {See 

SMITH, page 1, section II, wherein this reads over "color indexing by localized or regional color 
provides for partial or sub-image matching between images"}, wherein: 

the first image is associated with a first code {See smith, page 2, subsection b, wherein 

this reads over "[o]nce the image regions are identified, each region is characterized and 
represented using a feature set"; and page 3, subsection B, wherein this reads over "[a] color 
histogram is a high-dimensional feature vector typically having greater than 100 dimensions and 
the comparison of histograms is computationally intensive"} associated with a first 
predetermined textual annotation {See BARBER, Figure 2; and C5:L56-C6:L26, wherein 
this reads over "Image n is decomposed into image characteristics including color, texture, size, 
shape and layout. Arbitrarily, images or image features may be associated with text 
keywords""}, 

the second image is associated with a second code {See smith, page 2, subsection b, 
wherein this reads over "[o]nce the image regions are identified, each region is characterized 
and represented using a feature set"; and page 3, subsection B, wherein this reads over "[a] 
color histogram is a high-dimensional feature vector typically having greater than 100 
dimensions and the comparison of histograms is computationally intensive"} associated with 
a second predetermined textual annotation {See barber, Figure 2; and C5:L56- 
C6:L26, wherein this reads over "Image n is decomposed into image characteristics including 
color, texture, size, shape and layout. Arbitrarily, images or image features may be associated 
with text keywords""}, 

the first code is descriptive of at least a region of the first image {See smith, page 

4, subsection A, wherein this reads over "[a] color histogram denotes the joint probabilities of 
the intensities of the three color channels"}, 

the second code is descriptive of at least a region of the second image {See 

SMITH, page 4, subsection A, wherein this reads over "[a] color histogram denotes the joint 
probabilities of the intensities of the three color channels"}, and 

the image analyzer uses a matching algorithm for image analysis of the first and 
second images along with a comparison of the first and second codes in 
determining if the first and second images are likely to compare favorably 

{See SMITH, page 10, wherein this reads over "[t]he color histogram intersection was proposed 
for color image retrieval" and "[cjolors not present in the user's query image do not contribute 
to the intersection"} , wherein: 

the matching algorithm comprises a plurality factors related to matching that 
affect how the matching algorithm operates {See smith, page 10, wherein this 

reads over "Four image retrieval techniques were used in the queries: (1) euclidean 
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histogram distance, (2) histogram intersection, (3) quadratic histogram distance and (4) 
color sets"^ 

the plurality of factors comprise at least one of parameters, characteristics, 

and/or constraints {See SMITH, page 10, wherein this reads over "Histogram 
intersection ... the intersection of histograms h and g" and "Histogram Quadratic distance 
... the quadractic distance formula uses the cross-correlation"},. 

selection or exclusion of an image is analyzed to identify a subset of the 

plurality of factors {See SMITH, page ll, wherein this reads over "Query 1 ... the color 
set query was formulated in VisualSEek by simply drawing one purple region and by 
selecting best match in color and none for match in space" and "Query 1 ... In order to 
return all 83 sunsets using color histograms ... using color sets all sunset images are 
found"} , and 

the matching algorithm is automatically tailored to the subset of the plurality 

of factors {See SMITH, page 3, wherein this reads over "the user may search for images 
using both global and local features"}. 

While SMITH may fail to expressly disclose a method wherein code is associated with a first 
predetermined textual annotation, BARBER discloses an invention wherein textual data is associated with 
specific features and attributes of an image. Therefore, it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to modify the above invention suggested by SMITH 
with the invention as disclosed by BARBER. 

One of ordinary skill in the art would have been motivated to do this modification so that 
the predetermined textual annotations may be used in the retrieval and comparison of images. 

8. As per dependent claims 2 and 24, SMITH, in combination with BARBER, discloses: 

The system of claim 1 further comprising an input system coupled to one of said search 
engine and said image analyzer, said input system comprising at least one of: 

a graphical user interface {See smith, Figure 3}; 

a facsimile system; 

a camera system; 

a scanner; 

a network connection; and 
a video system. 

9. As per dependent claim 3, SMITH, in combination with BARBER, discloses: 
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The system of claim 21 wherein the matching algorithm defines at least one particular 
region of an image and at least one particular measurement to make on pixels within 
each of the at least one particular image region {See smith, page 4, subsection a, wherein 

this reads over "[t]he color histogram is computed by discerning the colors within the image and counting 
the number of pixels of each color"}. 

10. As per dependent claim 4, SMITH, in combination with BARBER, discloses: 

The system of claim 3 wherein the matching algorithm defines at least one measurement 
to make on one or more pixels in an image region neighboring the one particular 
image region {See SMITH, page 8, subsection E, wherein this reads over "[f]or each of the regions the 
generative color set c is recorded and the region area and location are measured"}. 

11. As per dependent claims 5 and 23, SMITH, in combination with BARBER, discloses: 

The system of claim 1 further comprising a storage device having at least one image 
stored therein coupled to at least one of said search engine, and said image analyzer 

{See SMITH, Figure 2}. 

12. As per dependent claim 21, SMITH, in combination with BARBER, discloses: 

The system of Claim 1 wherein said image analyzer is provided information specific to a 
particular application to modify a matching algorithm used in determining if the first 

and second images compare favorably {See SMITH, page 3, wherein this reads over "VisualSEEk 
supports query by color and texture and spatial-layout"}. 

13. As per dependent claim 36, SMITH, in combination with BARBER, discloses: 

The system of Claim 1, wherein the search algorithm is automatically tailored to a subset 
of possible image factors that can be automatically analyzed, whereby the image 
analyzer automatically learns which factors are important in matching images {See 

SMITH, page 3, wherein this reads over "for a local color region query the user indicates the locations of 
the color regions by positioning regions on the query grid"}. 

14. As per dependent claim 37, SMITH, in combination with BARBER, discloses: 
The system of Claim 1, wherein: 

the second image is part of a plurality of images, which are associated with a 

plurality of codes {See SMITH, page ll, wherein this reads over "the color set method was able 
to retrieve all 97 purple region images"}, 

the image analyzer compares the first code and the plurality of codes to find a subset 
of the plurality of images that compare favorably, wherein the second image is 

part of the subset {See SMITH, page 11, wherein this reads over "prior to the trials each of the 
3100 images was inspected and assigned a relevance to the purple region query"}. 

15. As per dependent claim 38, SMITH, in combination with BARBER, discloses: 
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The system of Claim 1 wherein at least one of the first and second predetermined textual 

annotations is human determined {See SMITH, wherein this reads over "retrieve all images that 
contain a significant purple regions somewhere in the image"}. 

16. As per dependent claim 39, SMITH, in combination with BARBER, discloses: 

The image processing system for processing images stored on the network as recited in 
claim 22, wherein the first code is textual and derived with manual determination 

{See SMITH, wherein this reads over "retrieve all images that contain a significant purple regions 
somewhere in the image"}. 

Response to Arguments 

17. Applicant's arguments with respect to claim rejections under 35 U.S.C. 102 have been considered 
but are moot in view of the new ground(s) of rejection. 

Conclusion 

18. Any inquiry concerning this communication or earlier communications from the examiner should 
be directed to PAUL KIM whose telephone number is (571)272-2737. The examiner can normally be 
reached on M-F, 9am - 5pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Tony Mahmoudi can be reached on (571) 272-4078. The fax phone number for the organization where 
this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system. Status information for published applications may be obtained from 
either Private PAIR or Public PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 
866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or 
access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Paul Kim 
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